Many researchers and educators have identified self-assessment as a vital aspect of professional self-regulation.1,2,3 This rationale has been the expressed motivation for a large number of studies of self-assessment ability in medical education, health professional education, and professions education generally. Unfortunately, the outcome of most studies would seem to cast doubt on the capacity for self-assessment, with the majority of authors concluding that self-assessment is, in fact, quite poor.4 In a recent article, Ward and colleagues suggested that this conclusion must be questioned because the methodologies used to evaluate self-assessment are fraught with methodological weaknesses.4 However, even studies that have attempted to address the weaknesses within the methodological paradigm have produced little evidence for effective self-assessment.5 Thus, the health professional education community is left with a conundrum that can only be resolved by deciding either that the conclusions of the studies are wrong, or that a critical premise underlying the concept of “self-regulation” in the professions is unsupportable. The current paper addresses this conundrum by arguing that there is a problem with the literature on self-assessment, and that this problem is more fundamental than a list of easily correctable methodological flaws. Rather, the roots of the problem in the self-assessment literature involve a failure to effectively conceptualize the nature of self-assessment in the daily practice of health care professionals, and a failure to properly explicate the role of self-assessment in a self-regulating profession. Until such an articulation of self-assessment is elaborated, it is difficult to know even which literatures might be informative in addressing this issue, and impossible to develop programs of research that operationalize the concept of self-assessment ability in a form that can be effectively studied. Thus, we will begin with a brief reflection on the various functions of self-assessment for a practicing health care professional and the manner in which these functions operate. The Purposes of Self-Assessment in Practice Self-assessment has been defined broadly as the involvement of learners in judging whether or not learner-identified standards have been met.6 While attractive due to their concise and encompassing nature, we fear that such simple definitions risk being misleading as they can cause underappreciation of the complexities of the construct. Self-assessment functions both as a mechanism for identifying one’s weaknesses and as a mechanism for identifying one’s strengths. Each of these mechanisms can be considered to have distinct, albeit complementary, functions. As a mechanism for identifying weaknesses or gaps in one’s skills and abilities, self-assessment serves several potential functions. First, in daily practice, the identification of one’s weaknesses allows the professional to self-limit in areas of limited competence. For example, in many circumstances the professional can quickly reject certain plans of action because she recognizes that she is unlikely to be able to complete the component tasks necessary to enact the plan. In other circumstances, a professional might recognize that he is “over his head” in a particular case and decide that it is time to recruit additional resources: to “look this up,” to obtain a consultation, to recruit additional support, or to refer the problem to another individual who is more competent in this domain. Second, in reflecting on one’s practice in general, the ability to identify weaknesses can serve the function of helping the professional set appropriate learning goals. That is, the traditional model of self-regulated continuing professional development presumes that an individual will select ongoing learning activities that fill professional gaps, but this presumes that the professional can effectively self-assess. Thus, in this role, the identification of weakness can help a professional to decide what must be learned. As a corollary to this, effective self-assessment is vital for setting realistic expectations of oneself, to avoid setting oneself up for failure. Thus, the identification of weakness also helps the self-regulating professional to decide what not to try learning, what should be accepted as forever outside one’s scope of competent practice. There is a complementary set of functions served by the ability to accurately self-assess one’s strengths. First, in daily practice, having a clear and accurate sense of one’s strengths allows the professional to act with appropriate confidence. For example, knowing one’s strengths provides the professional with the confidence to move forward on a fitting plan of action without inappropriate hesitation or trepidation. Similarly, it ensures that the individual will choose to persist on an appropriate plan of action in the face of initially negative feedback. The right path is not always smooth even if it is right, and early abandonment of an appropriate plan of action is as costly as selecting an inappropriate plan in the first place. Second, when reflecting on one’s practice in general, an appropriate assessment of one’s strengths ensures that one can set appropriately challenging learning goals, pushing the edges of one’s knowledge rather than choosing professional development courses that merely reiterate what one already knows. At the same time, by knowing one’s strengths, a professional can select learning objectives that are within her grasp, and therefore will be able to enjoy the motivational influence of attaining her goals and experience the satisfaction of a job well done. Together, then, the ability to accurately assess one’s weaknesses and one’s strengths generates a capacity for finding an effective balance both in daily practice and in setting personal learning goals. In daily practice, it generates a balance of confidence and caution, of persistence and flexibility, of experimentation and safety, and of independence and collaboration. In establishing learning goals, it generates a balance of learning enough but not too much, of starting neither too high nor too low, of knowing what to tackle and what to abandon. And in reflecting on accomplishments, it generates a balance of satisfaction and incentive, of self reward without self delusion. In order to fulfill these various functions, it seems that self-assessment must be effectively enacted in three forms: summatively, predictively, and concurrently. Enacting self-assessment summatively, a professional must reflect on completed performances both for the purposes of assessing the specific performance and for the purposes of assessing his abilities generally. When evaluating performance on a particular task, the professional can often assess the overall quality of the completed job as a question that may come in various forms. That is, the individual might ask how good this performance was relative to what she could have done; relative to what her peers might typically do; relative to the best that could have been done (a gold standard); or relative to some minimally acceptable standard. Alternatively, there are some situations where the mechanisms for objectively assessing the outcome are not immediately available, in which case the professional might ask herself how confident she is in the conclusion or outcome generated (is it right? will it stand up? could there have been a better solution given the situation?). The professional might then use her assessment of the specific task to draw summative conclusions about herself or her abilities in this domain generally. Again, such conclusions may be in absolute terms (am I good enough in this domain? am I minimally competent?) or in relative terms (am I average, above average, or below average, and against whom should I be comparing myself?). In drawing general conclusions about her abilities from a particular performance, the professional must also make determinations about whether this particular episode should be taken as an appropriate reflection of her general skills: were there extenuating circumstances that led to a particularly poor (or good) performance that might lead one to discount this outcome as reflective of overall ability? In addition to these summative functions, self-assessment must be used predictively. Professionals are constantly required to assess their likely ability to manage newly arising situations and challenges. In this predictive role, self-assessment leads to questions such as: Am I up to this challenge? Should I be starting this task (now, alone, in this way)? What are realistic goals for accomplishment in this context (what would I consider to be a good or acceptable outcome for me)? How much better might I imagine performing with some additional preparation and is the increased preparation worth the anticipated increase in performance? What additional resources should I recruit (either internally or from the outside) to complement my strengths and shore up my weaknesses? Finally, self-assessment plays a vital role in its concurrent mode of functioning. In this concurrent mode, self-assessment acts as an ongoing monitoring process during the performance of a task. It is self-assessment in its concurrent mode that leads to questions such as: Is this coming out the way I expected? Am I still on the right track? Am I in trouble? Should I be doing anything differently? Should I persist in the face of negative feedback from the situation (that things are not going the way I thought they would or as easily as I thought they would)? Do I need to recruit additional resources (internal resources such as attention or external resources such as advice/assistance)? Do I need to reassess my original goal or my original plan? Thus, self-assessment is a complicated, multifaceted, multipurpose phenomenon that involves a number of interacting cognitive processes. It functions as a monitor, a mentor, and a motivator through processes such as evaluation, inference, and prediction. Given this elaborated description of self-assessment, it is unlikely that simplistic questions such as “are health professional trainees effective self-assessors?” will lead to insightful discoveries about the nature and value of self-assessment. Rather, researchers must ask questions such as: On what basis do individuals make these decisions? What factors affect their reasoning? How fine tuned does the assessment need to be in order to be useful? A first step toward addressing these questions must be to determine who is already asking them and what insights we may borrow from their discoveries and reflections. Our search has led us to several literatures that seem particularly relevant: self-efficacy and self-concept; cognitive and metacognitive theory; social cognition; models of expert performance and the development of expertise; and the concept of reflective practice. In the following sections we will briefly touch on each of these literatures and suggest how they might inform our understanding of self-assessment. Our intent here is not to provide a systematic review of each literature, but to provide an overview of questions being addressed by researchers outside medical education that should inform our conception of self-assessment as a regulatory strategy. For each new literature we will define the area, provide examples of the issues under consideration, and then summarize the implications for self-assessment in the professions. We will end with a proposal for a program of research that has the potential to move the field beyond our current paradigm of repeatedly concluding that self-assessment is generically poor. Self-Efficacy and Self-Concept In studying the accuracy of self-assessments, education researchers in the health professions have tended to focus conceptually on what we have labeled the summative function – the ability to draw general conclusions about one’s skills or knowledge in specific domains: How well do I understand endometriosis? Am I able to communicate effectively with other members of the health care team? Practically, this has usually been operationalized in research studies as a request that students try to estimate how well they will/did perform on an immediately following/preceding task. Yet, there is an important distinction between general assessments of one’s ability in an area and the more specific question of how one did on a particular task. Researchers in the field of personality theory, for example, usefully distinguish between judgments of self-efficacy and the development of self-concept. Self-efficacy is the belief in one’s capabilities to recruit the resources and execute the actions required to manage prospective situations. Self-concept is the relatively sweeping cognitive appraisal of oneself that is integrated across various dimensions.7 Thus, self-concept beliefs are context free, generalized judgments of self-worth that involve cognitive self-appraisals independent of a specific task or goal (but not necessarily independent of domain). By contrast, self-efficacy is a context specific assessment of competence to perform a specific task or range of tasks in a given domain (i.e., an individual’s judgment of her capabilities to complete a given goal). Self-efficacy is, by its very nature, driven by an interaction between self-concept beliefs about one’s skills or abilities and the specific context in which those skills or abilities will be applied for the attainment of the particular goal. It is concerned with the contextually embedded orchestration of skills that lead to performance. Self-efficacy differs importantly from the concept of self-assessment as currently envisioned in the health professions education literature in that self-efficacy is not only influenced by direct and indirect feedback, but also influences the future performance of tasks (the choices we make, the effort we put forth, how long we persist when confronted with obstacles or in the face of failure). Thus, there is an important reciprocity between self-efficacy and success. Not only will success lead to a strong sense of self-efficacy, but self-efficacy will also lead to an increased likelihood of success. Self-efficacy beliefs are not merely passive reflections of performance, but part of a self-fulfilling prophecy that affects performance. As a result, there is an advantage to high self-efficacy beliefs even in circumstances where such beliefs may not be warranted by past performance. Clearly there is a logical disadvantage to continually overestimating one’s abilities, but this obvious disadvantage must be balanced with the value of believing that one can achieve more than one has in the past and that one can manage the challenges that one will face.8 As a result, researchers in the field of self-efficacy appear to be less worried about the “accuracy of self-assessment” and more worried about its impact on impending problem solving situations. They unconcernedly alter the situational self-efficacy of study participants through manipulations such as: varying the order in which people consider hypothetical levels of future performance,9 having subjects contemplate various positive or negative performance-related factors,10 altering the “anchor” values representing high or low levels of performance,11 or providing false performance feedback.12 Such manipulations regularly alter subjects’ expectations of success on future events within the context of the study, suggesting that subjects will take contextual information into account when judging (either explicitly or implicitly) the likelihood of future success on tasks within that context. Again, for researchers engaged in the study of self-efficacy, the important point to be taken from these studies is that “trivial” factors alter self-efficacy and can affect future performance.13 For them, the fact that one can radically alter an individual’s self-assessment of future performance appears to be simply taken for granted, rendering the question of “accuracy” somewhat nonsensical. Early on, Bandura provided a taxonomy of origins from whence information that would influence self-efficacy could be received.14 It included personal experience, vicarious experience, verbal persuasion, and physiological state. In addition, Cervone has argued that fundamental cognitive mechanisms (including common heuristics, as will be discussed in the next section) will influence the extent to which information from any given source will be weighed.13 In general, Cervone argues that self-efficacy judgments are not simply driven by an active, motivated distortion of facts in the service of ego protection (“hot cognition”), but rather that fundamental cognitive processes (i.e., those regularly used for a wide variety of judgment tasks – “cold cognition”) influence self-efficacy beliefs quite independently. Overall then, it appears that researchers in the self-efficacy literature offer several theoretical and methodological approaches that can inform research in self-assessment. They acknowledge, in fact presume, the instability and situational specificity of self-reflective judgments, they examine and explicitly manipulate the factors that affect these judgments, and they concern themselves with the consequences of these judgments for future behavior. Cognitive and Metacognitive Theory In contrast to the focus on “accuracy” in the self-assessment literature and the focus on “consequences” in the self-efficacy literature, cognitive psychologists interested in metacognition (knowledge of one’s own knowledge) tend to focus on delineating the mechanisms that allow us to mentally supervise and control the way in which we process information. Of particular interest for our purposes are questions of how people form metacognitive judgments, and what cues influence people’s judgments of how well they have learned something. It is a fundamental assumption of this work that we do not have direct introspective access to our own memories or knowledge base. Rather, just as we must infer others’ level of knowledge and motivations from their behaviors and other cues, so too we must use peripheral cues to make inferences about our own level of knowledge and learning. In fact, it is argued that our judgments of our own abilities are often based on the same inferential cognitive strategies, or heuristics, that we use to judge others. For example, the easier it is to process a piece of information, the more likely we are to judge that we will remember that information later (a fluency heuristic).15 Such heuristics are cognitive short-cuts that make us extremely effective and efficient at operating within a complex world despite our limited mental resources. However, they can also bias us in a way that leaves us susceptible to errors in decision making and, when applied to ourselves, errors in trying to identify our own strengths and weaknesses. Studies from this field suggest that, when trying to judge one’s ability in a domain or when trying to judge the likelihood of success on a task, the accuracy of these metacognitive judgments is dependent on the extent to which the apparent difficulty of learning mimics the actual difficulty of eventually retrieving the learned material from memory. For example, research demonstrates that, when people are trying to learn a piece of information (such as a list of words) for later recall, several factors affect their judgments of having succeeded in their learning efforts. Metacognitive judgments are more accurate if the repetitions of each word are spaced apart and interspersed with other words than if repetitions of each word are blocked together.16 People appear to use the cue of fluency (i.e., ease of understanding) in judging the extent to which they have learned material and, as such, overestimate the amount they have learned when fluency is increased by blocking repetitions together. Similarly, metacognitive judgments are more accurate when there is a delay between study of the words and efforts to recall during practice. In general, people overestimate their learning if the words are blocked or if recall follows too closely on study of the words, because these forms of the task are easier than the actual task they will eventually be expected to perform (recall after a long delay). The harder the retrieval task during the learning period, the better the predictions of the amount of learning that took place.17 Importantly, however, merely mixing the list and delaying recall during practice are often insufficient to improve metacognition if people are left to their own devices during learning. That is, in order to recognize one’s inability to recall the words it is necessary to actually try to recall them and make explicit mistakes in retrieval. Without these explicit errors as feedback, people continue to overestimate their ability to recall the words. Interestingly, participants are unlikely to spontaneously induce in themselves the failures that enable better judgments of learning. For example, judgments of learning tend to be more accurate after participants are forced to provide a response and produce the wrong word than if they are allowed to say, “I don’t remember.”18 This suggests that, without external pressure to do so, the participants did not try and fail, but rather simply did not try, and in doing so, missed an important cue that they might have used to improve their self-assessments. This finding is consistent with the higher correlations between performance and self-assessment seen in the health professions literature when judgments are elicited postperformance relative to preperformance.5 Taken as a whole, the findings from this literature emphasize the importance of moving beyond questions of “can people self-assess accurately” to ones that explicitly focus on the various factors that affect judgments of learning or knowledge or ability. In the absence of direct access to our mental states, we are forced to make metacognitive judgments based on a variety of internal and external cues. Metacognitive judgments tend to be more accurate when these cues accurately reflect the factors that affect subsequent performance,19 but there are many instances in which the cues used for judgments of learning lack predictive validity or worse yet, induce systematic discrepancies between predicted performance and actual performance.20 A better understanding of which cues are used and which ones should be used in health professional education contexts (as well as the impact these cues have on study habits) might better guide training strategies and improve our understanding of the concept of self-regulation. Some insight into the types of cues that are often misleading in real world situations (and reasons for the lack of insight into the inappropriate use of specific cues) has been gained from researchers working in the field of social cognition, the focus of the next section. Social Cognition Research in social psychology has led many to conclude that much of what we want to know about ourselves resides outside of conscious awareness.21 Each of us possesses an adaptive unconscious that guides much of our behavior, motivations, and feelings. This part of the mind is labeled unconscious because although we have privileged access to the contents (current thoughts, memories, and objects of attention), we do not enjoy such access to the mental processes that are engaged. We have a tendency to confabulate explanations for our behaviors,22 but these explanations are often inference-based and no more trustworthy than are introspections about the inner workings of our kidneys.23 This unconscious is adaptive because there are benefits to naivety. Most people believe themselves to be more popular, a better driver etc., than the average person.24 While it is logically impossible that we are all above average, at the individual level such positive self-deceptions can be beneficial in practice; individuals who maintain such illusions are less likely to be depressed and more likely to persist at (and succeed on) difficult tasks.25 Gilbert and Wilson talk of the psychological immune system, highlighting the great lengths we will travel to maintain a sense of well being, rationalizing and justifying threatening information.26 How we rationalize is somewhat idiosyncratic, but Gilovich, as one example, offers a number of mechanisms (some motivated, some inherent in fundamental cognitive processes) by which intelligent, thoughtful people can develop and maintain erroneous beliefs, many of which are relevant to an appreciation of what is required to accurately self-assess one’s own strengths and weaknesses.27 As one example, Gilovich describes gambling tendencies and presents evidence that counters the common belief that gamblers think they can beat the odds because they ignore or forget their losses. On the contrary, gamblers focus more attention on their losses and remember them better than wins. They maintain the belief that they are successful, however, by discounting the losses, focusing on the reasons why they should have won if not for some fluke event (e.g., the quarterback being injured). As a result, gamblers come to think of losses as “near wins” and thus maintain the belief that they can beat the odds. Learners likely find themselves in a similar situation. It is very easy to maintain an inaccurate perception of one’s own ability by making claims like “I knew the answer, but read the question wrong” or “Wow, I made a lucky guess in response to that question.” This tendency to discount conflicting information, combined with the rarity of corrective feedback increases the likelihood that flaws in reasoning will be reinforced. Given that the ultimate goal of self-assessment is actually to avoid such biased images of oneself, social psychologists suggest that it is necessary to look outward at one’s own behavior and how others react to it rather than simply reflecting inward.28 When reflecting on our knowledge and abilities we have a great deal of information available to us that is not available to anyone else (e.g., private knowledge/idiosyncratic theories), but our phenomenal capacity for discounting distortions that do not fit with our perception of reality can render illusory the feeling of triangulation that additional information provides, thereby resulting in a misleading feeling of confidence in the accuracy of our judgments.28 Sources of such illusions include a tendency we have to find more exceptions than truly exist, placing undue weight on apparently unusual factors,29 and being more likely than external observers to overlook situational influences on our actions, the tendency to do so being broadly recognized as the fundamental attribution error.30 This creates a paradox for self-regulating professionals in that it suggests one must systematically and intentionally elicit the views of others (both explicit opinion and implicit reaction) in order to fully develop an accurate impression of oneself. Without question the perceptions of others are also prone to distortions, but the more heterogeneous the sources of information, the less susceptible our self-concept might be to biased search for confirmation. This notion that self-assessment is insufficient for the evolution of accurate self-concept is consistent with the finding that peers tend to be better predictors of performance than do individuals rating themselves, both in health sciences,31 and social psychology.32 This view again raises questions about self-assessment quite distinct from the simple question of accuracy that has preoccupied self-assessment researchers in professional education. To what extent do health care practitioners seek out assessments from others? What prompts them to do so? How can we optimally supplement self-assessments with the views of others to create a coherent and appropriate sense of self? To what extent is coaching/mentoring/peer evaluation necessary/beneficial for such achievements to be reached? This latter question has been a major focus in determining the characteristics of expert performance. Models of Expert Performance and the Development of Expertise Some social psychologists have argued that the adaptive unconscious is largely a pattern detector whereas the conscious serves more as a fact checker.33 Taken broadly, this is also consistent with current models of proficient clinical reasoning, a construct being characterized as the flexible adaptation of multiple approaches to reasoning including both a nonanalytic, Gestalt-like, consideration of new cases, and a more carefully controlled (i.e., analytic) consideration of specific features.34,35 In the current context, the question of interest is what role, if any, does this conscious or analytic process of self-regulation play in the development and maintenance of expertise. In narrowly focusing the study of expertise on replicable elite performance, Ericsson and colleagues have been able to demonstrate on repeated occasions (and in diverse domains) the importance of deliberate practice – effortful, individualized training on specific tasks selected by qualified teachers.36 Deliberate practice is distinct from the enjoyable state of play (characterized as flow—giving up reflective control)37 and work (leading to immediate monetary and/or social rewards). Central to its definition is the presence of an instructor who can push students beyond their current ability by pointing to problems or novel approaches that are likely to go undetected if one relies solely on self-direction. In fact, notable by its absence in all domains of expertise except the health professions is an emphasis on self-directed learning. Contrary to popular belief, the role of early instruction and maxima
Search from 250 M+ research papersSearch from 250 M+ research papers
Just Published
Apr 30, 2024
Mari Starup + 2
Open Access